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This paper describes an exceptionally simple scheme for binary partial 
response signal formats of the form a, : ± a t _, (for I ^ 1, and a k = ±1). 
The receiver implements the maximum likelihood detector of the sequence 
a k assuming additive white Gaussian noise as the channel impairment. 
It is simpler and more efficient than the scheme recently described by 
G. D. Forney. 1 It is, however, not generalizable to multilevel signaling 
while still retaining its simplicity. 

I. INTRODUCTION' 

There has recently been considerable interest in using the inherent 
redundancy of the partial response signal formal to approach the 
error rate versus signal-to-noise-ratio performance equivalent to binary 
antipodal signaling. Forney 1 at the 1970 International Symposium on 
Information Theory discussed a simple decoding scheme which he shows 
to be asymptotically optimal for high signal-to-noise ratio for channels 
with white additive Gaussian noise. 

This paper describes a receiver for binary partial response signaling 
which is optimal for white additive Gaussian noise. This demodulator 
is much simpler than the equivalent two-level scheme of Forney. 
However, the extension to four or more levels seems to result in a 
scheme of much greater complexity than Forney's. In the first part 
of the paper we briefly review binary Class IV partial response signaling. 
Then we derive the optimal detection scheme for binary signaling which 
has a particularly simple implementation. A simple analysis of the 
memory requirements of the implementation follows. Finally we discuss 
some of the problems of extensions to multilevel signaling. 

II. A PARTIAL RESPONSE SYSTEM* 

The motivation for binary partial response signaling schemes is to 

* This section is almost entirely due to D. D. Falconer. 2 E. R. Kretzmer 1 and 
A. Lender 4 did the original work in this area and Lucky, Salz, and Weldon 5 have a 
good survey and summary. 
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allow transmission of two bits per cycle of bandwidth without requiring 
ideal boxcar niters. The train of signal waveforms is shaped so that 
inherent intersymbol interference does not affect decisions made by 
the receiver. 

Figure 1 shows a basic partial response signaling scheme transmitting 
\/T bits per second. Information bits (a k ) are represented by +ls 
and — Is. Signal shaping is done by the filter whose transfer function 
is Z(co). A "Class IV" partial response function X(co) and its associated 
sampled impulse response are shown in Fig. 2. This particular scheme 
is useful since it has no transmitted dc component. It is used in several 
existing and proposed partial response modems. 

The transmitted Class IV partial response signal s(t) can be rep- 
resented in terms of the sequence of samples (x k ) spaced at Nyquist 
intervals (T seconds) as 



s(t) = A 2 x * smc \TF ~ for 



(1) 



where 



sine (x) = 



and 



x k — a k a, k -2 , 



sin x 
x 



k = 1, 2, 



(2) 

When the information symbols a k take on values ±1, then the 
samples (x k ) have three possible levels: 0, +2, or —2. Thus the scheme 
would be expected to be more sensitive to noise than is a comparable 
binary antipodal scheme in which x k = ±1, and in which the trans- 
mitting filter's transfer function is a "boxcar." In fact, when independent 
hard decisions are made on each bit, it can be shown to require 3 dB 
higher signal-to-noise ratio in order to achieve the same error rate 
as the comparable binary antipodal scheme. A more efficient conven- 
tional partial response configuration which is 2.1 dB worse than binary 
antipodal 5 is to provide a matched filter at the receiver by replacing 
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Fig. 1 — Partial response system. 
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Fig. 2 — Class IV partial response: (a) spectrum; (b) sampled impulse response. 

X(u) at the transmitter by X(oj)' at the transmitter and X(a>)* at 
the receiver. Other classes of binary partial response systems are 
worse than the ideal binary antipodal scheme by various amounts 
(see Table 4-2, page 91, in Ref. 5). 

With a k = ±1, x k = a k — a k - 2 is a sequence of three-level signals. 
However not all the sequences are possible! For example, if a k = +1, 
x k = +2 or and if a k = — 1, x k = or —2. All the schemes described 
use this inherent redundancy to win back the 2.1-dB loss alluded to 
previously. Finally we note that all partial responses of the form 



X k — &k Ofk-l ) 



l^ 1, 



produce I noninteracting streams of x k s. For I = 2, the even x k s and 
the odd x k s are entirely independent. A scheme for I = 1 can be used 
for any I ^ 1 by time sharing its operation with the other independent 
streams of x k s. This observation allowed Forney to assert the applic- 
ability of his scheme for all I. It also allows us to consider only 1=1. 



III. DERIVATION OF OPTIMAL RECEIVER 

The receiver that we develop is to be optimal for additive white 
Gaussian noise and the signaling format 

x k = a k — a k -i (3) 

with 

a., = 1. 

A simple way to describe the sequence of x k s resulting from a sequence 
of a k s is given by the trellis in Fig. 3. The branches of the trellis are 
the x k values and the nodes are the a k values. The upper nodes are +1 
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and the lower values are — 1. We trace a particular sequence of x k s 
by following the branches joining the nodes for the appropriate a k . 
For instance if we have the sequence starred (*) in Fig. 3, namely • • • , 



NODE 




k-i 



k k + i 

Fig. 3— Signal trellis. 



a*-i = 1» O-k = — lj 
output sequence • • • 



= — 1, a k , 2 = 4-1, 



then we have the 



x k 



= —2 



x k+i = 0, x k+2 = 4-2, • • • by following 



the appropriate branches. Notice that we have the capability of de- 
scribing any possible sequence of x k s using Fig. 3. Further we note 
that any node has two branches leading to it and away from it. 

The channel is assumed to add white Gaussian noise n k , with density 
N(0, a 2 )* giving a received signal y k = x k + n k . It is well known that 
the maximum likelihood receiver chooses the infinite sequence of d k s 
which maximize 



-. oo -t oo 

^ fc-o * *=o 

d-x = 1, 



(4) 



for a given sequence of y k s. The d k s are the estimates of the transmitted 
sequence \a k }. While it is clearly impossible to maximize (4) directly, 
it is possible to maximize (4) sequentially. We note that we can rep- 
resent all possible sequences of d k by paths in the signal trellis in Fig. 3. 
We also note that we can represent all possible sums in (4) as the 
result of paths through a trellis. We then obtain the trellis in Fig. 4. 
When d k and d h+1 are of the same sign, the branch contributes to 
the sum in (4) but when d k = 4-1, 4-i = — 1, it contributes y k — 1 
to the sum. Similarly when d k = —l,d k -i = 4-1, the branch contributes 
—y k — 1 to the sum. We say that a specific sequence of d k s through 
the trellis describes a path. All paths must have d k = +1 or 4 = — 1. 



* N(a, b) is the Gaussian density with mean a and variance b. 
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Fig. 4 — Received signal trellis. 
For all those paths with d k = -f-1 we can write (4) as 
\o 11 Vi(Ai ~ ^i-i) - 7 S («' - «J-0 f 

\4 1-0 * 1=0 J 

where 

4 = 1. 

Thus it is necessary that any path with d k = 1 and which maximizes 
(5) also maximizes the first bracketed sum in (5) . But this first bracketed 
sum in (5) depends only on {d„ , • • • , a*-i ) and this portion of the path 
can be chosen independently of the rest of the path. Define 



jt = max \- Y, ViiAi - «'-i) — 7 E (<*i — d,^) 2 }- 

nil Diiths \& l-=0 * 1=0 / 

wi th l'n = l 

We similarly define, for the best path leading to d k = — 1, 
/; = max \- J2 yiifii - «i-i) " 7 E («! " aj-i) 2 r 

all paths \6 *-0 * 1=0 / 



(6a) 



(6b) 



itha*--l 



Finally we see that there are only four branches from the Zeth to the 
(k + l)st node. Hence, if we have the best path to d k = ±1, then 
at d k + i = +1 we must choose between only two paths, the one coming 
from d k = +1 having a value j + k and the one from d k = — 1 having 
a value fl + y*+i — 1- The best path is obviously the one with the 
largest value. Thus we have 

f /* (+ PATH) M . 

]Ux = max J lk v (7a) 

I/* +y* + i - 1 (-PATH). 
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Similarly we have the best path to d k+ i = — 1 as the solution of 

4- ™v ]/* - *« - l (+ PATH ) an 

f k+1 = max ^ (7b) 

{ U (- PATH). 

Thus at any point in time we have two paths, one of which must be 
the beginning of the one which optimizes (4).* We say we are not 
merged at (fc — 1) if we still have two paths left at A;. Figure 5 shows 

+ i +i 
-o ^ 



^- 




(k-i) (k) (k-i) 

(a) lb) 

Fig. 5 — Possible nonmerge paths. 

the only two possibilities to remain unmerged. For the (+) path to go 
to d k+1 = +1 and the (— ) path to go to d k+l = — 1 (Fig. 5a) we need 
both 

f, ~n- V»+i + 1^0 from (7a) 

and 

-(/* - fl) + Vw + 1^0. from (7b) 

Thus we require 

-l ^ n - n - y M ^ i. (8) 

For us to remain unmerged on the "crossover" path of Fig. 5b we 
require 

[and « + tf-*«>l I. 

(/* -1D -y* + i < -iJ 

This is clearly impossible. Thus we remain unmerged if and only if 
(8) is true. Hence when (8) is true, we are unmerged and the most 
likely path from the + node leads to the + node, and the most likely 



* The formulation and solution of this problem is a simple example of Dynamic 
Programming and an application of Bellman's Principle of Optimality. 6 This may 
also be considered as a simple example of the Viterbi Algorithm which was shown 
by J. K. Omura 7 to be equivalent to Dynamic Programming. Finally, the identi- 
cal formulation and solution to this problem was also obtained independently by 
H. Kobayashi 8 and M. Segal (unpublished). 
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path jrom the — node leads to the — node. We also note that 

n - n - & + i > i (9) 

implies both best paths came from & k - x = +1 [a (+) merge] and 

n - n - y*+* < i (io) 

implies both best paths came from d*-i = — 1 [a (— ) merge]. Finally 
we see that all decisions as to merge or not are based on ft — 1k and 
not on either separately. We then define 

A. = n - n- 

Subtracting (7b) from (7a) and noting (8), (9), and (10) gives 



k +1 + 1, A k - y k+1 > 1 (+ MERGE) at k 

A t+1 = A t , -1 < A t - y k+1 < 1 (NO MERGE). 

4/ ft+1 - 1, A, - y k+1 < -1 (- MERGE) 



(11) 



The optimal receiver implements (11). We see that while unmerged, 
A* remains the same. Only the testing to see if we have finally merged 
depends on the incoming data. The value of A* while unmerged is 
just that resulting from the two paths leading from the most recent 
merge. Thus if the most recent merge was (-}-) at node I — 1 then 
A fc = y, + 1 for k ^ I and no merge. Between merges, only two se- 
quences are possible, either {1, 1, 1, ••• } or {—1, —1, —1, ••• }. 
Hence in our implementation all we have to do is save Aj and the 
location of the most recent merge. Since we will be placing our data 
in a storage register prior to outputing it, we must decide which of 
the two between-merge sequences should we store. Obviously, the 
best is the most likely of the two. 

We remember that after a (4-) merge at the (fc — 1) node 

A* = y k + 1. 

If the (+) merge is correct then 

Vk = a k — a k -! + n k 
= — 1 + a k 4- n k 
giving 

A t = a k 4- n k . 

Since a k = ±1, then A* = 1 4- n k if the transmitted path leads to 
the 4- node and A fc = — 1 4- n k if the transmitted path leads to the 
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— node. Hence the determination of the most likely path leading from 
the (k — 1) node is a binary hypothesis testing problem. The solution 
is to say the most likely path leads to the + node if A* > and the 

— node if A k < 0. 

For a correct — merge, the test is identical. The most likely path 
initially then is the one leading to sgn(A*). If when we finally merge, 
the sign of Aj at the merge point is the same as the merge, the most 
likely path is the same as the most likely path initially chosen on the 
basis of A* ^ 0. 

IV. IMPLEMENTATION 

An implementation is suggested by the flow diagram of Fig. 6. 
We suppress the subscripts. The newly received signal is y and the 
previously stored difference is A. The decoded data are stored in a 
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Fig. 6 — Flow diagram for xt = at — Ot-i. 
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register of length AT + 1. d„ is the most recent decision and d N is the 
bit about to be outputed. There is also a pointer, p, which indicates 
where the first data bit after the most recent merge point is located 
in the data register. The equations implemented are those of (11). 
The values of d a , ■ • ■ , d p are those of the initially most likely path 
as described in Section III. Referring to Fig. 6; when a new value 
y is obtained, we subtract it from the stored difference A calling this 
sum 2. We then check to see if a merge has occurred according to (11). 
If z > 1 then we have a + merge, if z < —1 we have a — merge, 
and if — 1 ^ z ^ 1 we have no merge. If a merge has occurred, then 
we will eventually replace A by y + 1 for a + merge and y — 1 for 
a — merge. We thus let a = ±1 for a ± merge. If the most likely 
path is actually the one we have been saving, then they must agree 
at the merge. We check this by finding out whether sgn A is the same 
as the merge value ±1. If it is, then we have saved the most likely 
path. If it does not agree, the most likely path is the complement 
of the one we saved up to the most recent merge point p. We then 
complement d„ , • • • , d p . After we have our data set up, we replace 
A by its new value y + a, and reset the pointer p to 0. At this point 
we shift the register and place d = sgn A. 

If there is no merge, then life is simpler; A is the same and the pointer 
is advanced by 1; the register is then shifted and d = sgn A. We are 
now ready for a new piece of data. Figure 7 shows a possible imple- 
mentation of the above flow diagram. 

V. BUFFER OVERFLOW (P > N) STRATEGY 

Since we are saving the most likely sequence, we just output the 
buffer and keep p = N. If when we merge, we do indeed have the 
most likely sequence, then all is fine. If the most likely sequence is 
not actually held, then we complement the entire register. Although 
we have sent some suboptimally detected bits, this appears to be the 
best strategy. We could save ourselves most of the problem if we 
differentially encoded ("PRECODED" 5 ) the data. This means we 
would let 

X k = a k — d k - x 

where 

a>tfl>k-\ = o-k • 

Under these circumstances, a single suboptimal decision in the decoded 
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Fig. 7 — Possible implementation for x k = a k — a*_i. 

a* path results in two errors in the a k path. However, if we hold the 
complement a k path after a merge point rather than the most likely 
path, we only make a single suboptimal decision in decoding the a k s 
rather than a possibly long burst. If we now complement the entire 
register, then we make an additional single error for the data bit af- 
fected by both d N and d N+l because d N was complemented and d N+l 
was not. If we do not complement the entire register, then the same 
phenomenon occurs at the next merge point. In both cases we make 
only two single suboptimal decisions whenever the buffer overflows. 
This obviously makes differential encoding of the data advisable. 



VI. ANALYSIS OF BUFFER SIZE 

In this section we determine the approximate probability of overflow 
of a buffer of length (N + I). If we have a (+) merge then 

(12) 



and for a ( — ) merge 



A = y n + 1 



A = y — 1 



(13) 



where the "0" subscript refers to the merge position. Because y is 

JV(-2, a 2 ) for a +- transition, iV(0, a 2 ) for a ++ or transition, 

and N(+2, a 2 ) for a — + transition (see Fig. 3) we have, for a correct 
decision, substituting into (12) and (13), that A is AT(1, a 2 ) for a transi- 
tion leading to el + node and A is JV( — 1, <r 2 ) for a transition leading to 
a — node for both types of previous merges. We know we remain 
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unmerged for N transitions if 

-1 ^ A - y, =g 1 1= I, ■■■ ,N. (14) 

Since the + + , H — , — h transitions are all equally likely, we 

have A — y t with the densities 

A — i/i — N(+3, a 2 ) with probability 1/4 (15a) 

A - yi ~ JV(+1, <r 2 ) with probability 1/4 (15b) 

A - y t ~ N(-l, a 2 ) with probability 1/4 (15c) 

A - Vi ~ iV(-3, a 2 ) with probability 1/4. (15d) 

For small a 2 (large signal-to-noise ratio) both (15a) and (15d) lead 
to (14) not being satisfied with a very high probability. We can ignore 
these two events. (15b) and (15c) both correspond to y t ~ N(0, a 2 ) 
and occur together with probability 1/2. Because of the symmetry 
of (14), (15b), (15c), and y t we can write the probability of no merge 
for at least N nodes as 

P(N) S Pr(-l g A - y t £ 1, I = 1, ■ • • , N) 

„ r i (a - 1) 2 [i r x - A i ^,/fc..?,. 

-Lvt^ exp - ~^~ h L. v*r ff e dy \ dA - 

Now if o- 2 is small then A is concentrated about 1 and the limit — 1 — A 
can be replaced by — oo . We then have 



™ « ST £ vb e ™ DC vfc/""" *T *■ 



(16) 
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we write (16) as 







P(iV)= (iJ fjQ(x)YdQ(x) 
N + 1 \2/ 



(17) 
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To obtain (17) we have really used only the fact that the distributions 
of A and y are translates of each other and that they are symmetric 
about their mean value. Equation (17) was also derived independently 
by Kobayashi. 8 

The Forney scheme 1 has a probability of buffer overflow of 2~ N . 
Equation (17) indicates a factor of N improvement in this case. For 
N = 20, (17) gives P(N) = 4.5 X 10~ 8 . 

VII. GENERALIZATIONS 

The most obvious generalization we would like is to four-level 
signaling. We can obtain a signal trellis in the same way as in the 
binary case but now we have four nodes at each time instant. Again 
we can write, using the same arguments as before, equations equivalent 
to (7) . However, all the special structure which led to the exceptionally 
simple results of (11) seems to be missing. Instead of only one set 
of no-merge paths as indicated by Fig. 5a, we have many. Instead 
of only one possible way for either a + or — merge to occur, we have 
several. It also appears that all four possible paths through the trellis 
must be kept. In short, for a four-level signaling, Forney's scheme 
seems to be the simplest but for binary signaling, the one described 
here is best. 

VIII. CONCLUSIONS 

The system here is applicable to partial response signaling with 
binary data of the form 

x k = a k ± a k ^i for all I ^ 1. 

Differential encoding of the data is helpful to reduce the effects of 
buffer overflow. The extension to multilevel signaling destroys the 
beauty and simplicity of the binary scheme. 
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